Papers with byte pair encoding
Stochastic Tokenization with a Language Model for Neural Text Classification (P19-1)
Copied to clipboard
| Challenge: | Sentences segmented with words or subwords can be difficult to perform text classification tasks. |
| Approach: | They propose a method to learn tokenization and text classification simultaneously to address these problems. |
| Outcome: | The proposed method improves on sentiment analysis in Japanese and Chinese using tokenization and text classification models. |
Data Augmentation for Voice-Assistant NLU using BERT-based Interchangeable Rephrase (2021.eacl-main)
Copied to clipboard
| Challenge: | a data augmentation technique is used to boost performance on spoken language understanding tasks. |
| Approach: | They propose a data augmentation technique based on byte pair encoding and a BERT-like self-attention model to boost performance on spoken language understanding tasks. |
| Outcome: | The proposed method performs well on domain and intent classification tasks for a voice assistant and in a user-study focused on utterance naturalness and semantic similarity. |
Lifelong Learning of Topics and Domain-Specific Word Embeddings (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing lifelong topic models focus on indomain text streams in which each chunk only contains documents from a single domain. |
| Approach: | They develop a lifelong collaborative model that uses non-negative matrix factorization to learn topics and domain-specific word embeddings. |
| Outcome: | The proposed model can learn topics and domain-specific word embeddings from a lifelong collaborative model. |
Wine is not v i n. On the Compatibility of Tokenizations across Languages (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models are used for performance and memory constraints, but there is little work that investigates the compatibility of tokenizations across languages. |
| Approach: | They propose a compatibility measure that reflects compatibility of tokenizations across languages. |
| Outcome: | The proposed measure prevents incompatible tokenizations, e.g., “wine” in English vs. “v i n” in French, which make it hard to learn good multilingual semantic representations. |
The Challenges of Optimizing Machine Translation for Low Resource Cross-Language Information Retrieval (D19-1)
Copied to clipboard
| Challenge: | Existing studies do not investigate the effectiveness of MT metrics in predicting performance of downstream IR models. |
| Approach: | They examine the relationship between MT performance and IR quality in a CLIR-based system . they find that the choice of IR collection can significantly affect MT tuning decisions . |
| Outcome: | The proposed model can predict CLIR performance better from MT quality, the authors show . the proposed model is based on a BLEU-based model with a bag of words constraint . |